Determining dissimilarities between digital maps and a road network using predicted route data and real trace data

ABSTRACT

A network system determines dissimilarities between a digital map and trace data of a road network in an area as service providers and service requestors coordinate service using the road network in the area. To determine dissimilarities the network system can determine a suggested route, determine a predicted route, receive executed trace data, and compare the predicted route data to the executed trace data for the suggested route. The network system may aggregate trace data when determining a dissimilarity. The network system can quantify the differences between traces to determine dissimilarities. Quantifications can include, ratios, bounds, and scores. The network system can determine and alternate route if a dissimilarity indicates that the state of a road segment has changed (e.g., from “open” to “closed”). The network system can modify guidance instructions if a dissimilarity indicates that a guidance instruction is misleading.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of co-pending U.S. Application No.16/376,940, filed Apr. 5, 2019, which claims the benefit of priority toU.S. Provisional Application No. 62/797,108, filed Jan. 25, 2019, thedisclosures of which are hereby incorporated by reference herein intheir entirety for all purposes.

BACKGROUND Field

This description relates generally to determining dissimilaritiesbetween maps and ground truth data for a road network and moreparticularly to using route differences between suggested routes andexecuted routes to determine a dissimilarity.

Description of the Related Art

Digital maps are often used to provide route guidance. Map datatypically includes features such as roads, bodies of water, points ofinterest, and the like, which are used to render the maps. Map data alsotypically includes information about how the map can be traversed-forexample, a map that includes road networks typically has associated datadescribing how the road network can be used, including direction oftravel (one way, two way), whether turns are permitted at variousintersections, etc.

As features such as roads are added to and removed from the roadnetwork, and as rules for how the network can be traversed are changed,the map data becomes stale and requires updating to remain effective.

SUMMARY

Dissimilarities between a digital map and a road network are problematicin transport service coordination. For example, dissimilarities cancause a network system to generate a suggested route that cannot betraversed by a transportation vehicle operated by a service provider whois providing transport service to a service requestor. Additionally,fixing dissimilarities between a map and an actual road network isdifficult. More particularly, ensuring that a map accurately reflects anever-evolving road network in an area is technically challenging.

A network system determines dissimilarities using predicted route dataand executed trace data for a suggested route. Predicted route data istrace data (i.e., position, velocity, acceleration data, etc.) expectedto be received from service providers and service requestorscoordinating transport service using the suggested route. Realizedroutes are trace data actually received from service providers andservice requestors coordinating transport service and provided thesuggested route. The network system may aggregate the trace data (bothpredicted and executed) to more accurately determine dissimilarities.When predicted route data and realized route data are different, thisdifference can be indicative of a dissimilarity. The network system canquantify the differences between the predicted route data and executedtrace data in a variety of manners. Based on the quantification of thedifferences, the network system can determine whether a dissimilarityexists, and what type of dissimilarity it is.

In one embodiment, the network system quantifies differences bygenerating a path from the trace data. A path is a representation of anumber of road segments and nodes expected to be traversed and/oractually traversed for a suggested route. The network system canquantify the differences between predicted paths and expected paths bycalculating expectation ratios, bounds, and state scores. Based on thestate scores, the network system can determine if a path, or a roadsegment in a path, includes a dissimilarity.

The network system may determine an alternate route if the state of aroad segment has changed based on a state score. For example, a statescore may indicate that a road state is “closed” or “open.” If thenetwork system detects a dissimilarity indicating that a road segmentchanged from “open” to “closed”, the network system may generatealternate routes that avoid that road segment.

The network system may modify guidance instructions for a suggestedroute to improve transport service coordination based on a state score.For example, a state score may indicate that a guidance instruction fora suggested route is misleading and causing service providers totraverse an incorrect route rather than a suggested route. In response,the network system may modify the guidance instruction such that fewerservice providers traverse the incorrect route.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of an environment for determining dissimilaritiesbetween maps and a current road network using trace data from clientdevices travelling suggested routes, according to one embodiment.

FIG. 2 is a process for determining a dissimilarity between a roadnetwork and a map using a route difference determined from realizedroutes received from client devices provided with suggested routes,according to one example embodiment.

FIG. 3 illustrates a network system, according to one exampleembodiment.

FIG. 4 illustrates a client device, according to one example embodiment.

FIG. 5 illustrates a map of a road network including road segments,nodes, realized routes, and predicted routes that can be used todetermine a route difference, according to one example embodiment.

FIG. 6 is a flow diagram illustrating a process for determining analternate route for a transport service based on route differences,according to one example embodiment.

FIG. 7 is a flow diagram illustrating a process for modifying guidanceinstructions for a transport service based on route differences,according to one example embodiment.

FIG. 8 is an illustration of an example machine for reading andexecuting instructions from a machine-readable medium, in accordancewith some embodiments.

The figures depict various embodiments for purposes of illustrationonly. One skilled in the art will readily recognize from the followingdiscussion that alternative embodiments of the structures and methodsillustrated herein may be employed without departing from the principlesdescribed herein.

DETAILED DESCRIPTION

The figures and the following description relate to various embodimentsby way of illustration only. It should be noted that from the followingdiscussion, alternative embodiments of the structures and methodsdisclosed herein will be readily recognized as viable alternatives thatmay be employed without departing from the principles of what isclaimed.

System Environment

A digital map (“map”) in a network system can be used to facilitateservice coordination between two parties operating client devices withina system environment. For example, a network system may be used tocoordinate a transportation service between a service provider and aservice requestor. Maps in the network system may be used for manypurposes within this context. For example, maps may be used whendetermining routing information, relative position, travel timeestimations, fare calculations, guidance instructions, road networkstates, etc.

Maps used by a network system, generally, include a road networkrepresenting an area in which the transportation service is beingcoordinated. Maps may be obtained from a third party or created in someother manner. However, in some cases, maps may have been createdinaccurately. In addition, over time, the road network in theenvironment may change and the maps are no longer an accuraterepresentation of that environment. In such cases, routes that aresuggested for a transportation service may be inaccurate. For example, asuggested route may include a turn that is illegal, a road segment thatis closed, etc. As such, a service provider may not be able to follow asuggested route determined from the map for the transportation service.In these cases, trace data from client devices traversing suggestedroutes-that is, data representing the actual movement of a client deviceas it traverses a suggested route through the road network-can be usedto determine a difference (“dissimilarity”) between the road networkrepresented in a map and the current road network. In other words, adissimilarity is a determined difference between a physical road networkand its digital representation in a map. Determined dissimilarities maybe output as a data structure for verification, used to improve guidanceinstructions, and/or suggest alternate routes.

FIG. 1 is a diagram of an environment 100 for determiningdissimilarities between maps and the current road network using tracedata from client devices travelling suggested routes, according to oneembodiment. The environment 100 includes a service requestor clientdevice 110A, a service provider client device 110B, a network 120, and anetwork system 130. FIG. 3 is a more detailed illustration of a networksystem 130, according one example embodiment. FIG. 4 is a more detailedillustration of a client device 110, according to one exampleembodiment.

The service requestor client device 110A and the service provider clientdevice 110B are configured to obtain trace data as devices 110coordinate transport service using a road network in the environment100. Trace data can include position, velocity, acceleration, or anyother service, geographic, or location-based information. Generally, aclient device 110 gathers trace data from sensors on the client devices110 (e.g., a global positioning system, accelerometers, etc.). In oneembodiment, client device 110 associates gathered trace data with thecurrent position of client device 110 as it traverses the current roadnetwork and sends the trace data to network system 130. Accordingly,because the trace data depicts actual movement through the road network,it provides a current representation of the road network in the area.

Network system 130 facilitates service coordination between devicesusing maps of the road network in the environment 100. As such, networksystem 130 includes a datastore that stores historical maps of the roadnetwork in the area such that network system 130 can reference the roadnetwork when coordinating transport service. In one embodiment, the roadnetwork may be represented in the network system 130 as sets of roadsegments connected by nodes and identified by coordinates. Further, theroad network includes a connection ruleset that defines allowable waysfor the road segments and nodes to be connected (e.g., direction oftravel, speed, etc.). The road segments, nodes, and connection ruleset,in aggregate, form a representation of a road network in the geographicarea of the environment 100.

The network system 130 may determine a suggested route from a firstlocation to a second location when coordinating a transport service. Forexample, the network system 130 can access a map of the road network inthe environment 100 and generate a suggested route including theappropriate road segments, nodes, and connection information (“routeinformation”) to allow a service provider to transport a servicerequestor from the first location to the second location using the roadnetwork. A suggested route may also include the route information forany other routes suggested during transport service. For example, aservice provider may begin providing transport service using an initialsuggested route. During transport service, the service provider isprovided a secondary suggested route and begins to provide transportservice for this according to the secondary suggested route. In thisexample, the suggested route includes the route information for both theinitial suggested route and the secondary suggested route.

In some cases, trace data received from client devices 110 traversingsuggested routes (“realized routes”) is dissimilar from trace datapredicted for a client device for that suggested route (“predictedroutes”). In other words, realized routes include route informationcollected by client devices 110 coordinating transport service provideda suggested route. Similarly, predicted routes include route informationexpected (or predicted) to be received from client devices 110coordinating transport service provided the suggested route. When apredicted route and a realized route reflect different route information(“route difference”), the route difference may indicate that a serviceprovider did not wholly adhere to the suggested route when providingtransport service. In some cases, a route difference may not indicate adissimilarity. For example, a service provider may prefer an alternateroute to the suggested route, a service provider may miss a turn, etc.

Network system 130 can match trace data from predicted routes and/orrealized routes to road segments of a map. For example, a serviceprovider provides transport service by travelling down a first roadsegment, turning left onto a second road segment, and turning right ontoa third road segment. All of the road segments correspond to a roadsegment in a map. The realized route for the transport service includesa group of GPS coordinates representing the coordinates of the serviceprovider as it traverses the road segments. Network system 130 matchesthe group of GPS coordinates to corresponding road segments in the map.For example, a first subset of the coordinates represents the first roadsegment and are matched to the first segment in the map, a second subsetof the coordinates represent the second road segment and are matched tothe second segment of the map, etc. In this manner, a route (suggested,realized, predicted) may be represented by map matched road segmentsrather than, or in addition to, trace data representing the route.

Network system 130 can determine a dissimilarity using route differencesdetermined from predicted routes and realized routes of client devicescoordinating transport service. Network system 130 can generate analternate route for a suggested route, determine state changes for roadsegments, generate corrective guidance instructions, and/or indicateincorrect guidance instructions for a suggested route based on adetermined dissimilarity and route difference.

As an example, FIG. 2 is a process 200 for determining a dissimilaritybetween a road network and a map using a route difference determinedfrom realized routes received from client devices 110 provided withsuggested routes, according to one example embodiment.

To begin, network system 130 determines 210 a suggested route for atransport service from a first location to a second location using a mapof a road network stored by network system 130. The suggested routeincludes route information that allows a service provider to transport aservice requestor from the first location to the second location.Network system 130 also generates 220 a predicted route for thesuggested route that includes the route information of the suggestedroute. Generally, a suggested route includes the road segments and nodesfor a transport service, while a predicted route represents trace dataexpected to be received from a client device traversing the suggestedroute. So, for example, a predicted route may include a set ofcoordinates representing the road segments expected to be traversed onthe suggested route. The predicted route may also include timeinformation for the set of coordinates. In some configurations, thepredicted route may be the same as the suggested route and only includethe segments and nodes of a suggested route.

As an example, Dan (i.e., the service requestor) requests transportservice from her house to a movie theater using her client device 110A.The network system 130 receives the request and facilitates theproviding of transport service to Dan by Christine (i.e., the serviceprovider) from her house to the movie theater using a transportationvehicle. Network system 130 accesses a map of a road network includingthe house and the movie theater and determines 210 a suggested route.The suggested route includes route information that allows Christine totransport Dan from her house to the movie theater. Network system 130transmits the suggested route to Christine’s client device 110B vianetwork 120. Network system 130 also determines 220 a predicted routefor the suggested route that includes route information for thesuggested route from Dan’s house to the movie theater. The predictedroute includes a set of coordinates approximating trace data expected tobe received by Christine and Dan as they traverse the road network.Network system 130 may also include an abstraction of the approximatedtrace data at the level of road segments and nodes similar to thesuggested route.

Client devices 110 traversing routes within the road network obtainrealized routes. The realized routes include route informationdescribing the route that each service provider actually takes whenproviding transport service. Thus, when a client device traverses asuggested route the realized route reflects route information for thesuggested route and when a client device does not traverse the suggestedroute it reflects route information for an alternate route. The clientdevices 110 transmit the realized routes via the network 120 and thenetwork system 130 receives 230 the realized routes. As an example,Christine is providing transport service to Dan in a transportationvehicle. While providing transport service, Christine is traversing theroad network from Dan’s house to the movie theater using, at thebeginning of transport service, the suggested route received from thenetwork system 130. Christine and/or Dan’s client device 110 obtains arealized route reflecting the route that Christine traverses from Dan’shome to the movie theater and transmits the realized route to networksystem 130. In this example, after some time, Christine and Dan take adetour from the suggested route because a recent construction project inthe area has started and the suggested route is no longer accurate. Inother words, Christine and Dan begin taking an alternate route. Thus, aspart of normal trace data collection, Christine and Dan’s client devices110 obtain a realized route reflecting the detour and alternate route.

Network system 130 determines 240 a route difference between a receivedrealized route for a suggested route and the predicted route for thatsuggested route. For example, Christine and Dan’s realized routeindicated that they took a detour to avoid the construction whenprovided with the suggested route. As such, realized routes from theirclient devices 110 reflect a route difference between the realized routeand the predicted route for that suggested route. The route differencecan indicate which road segments, nodes, and/or connection informationare different between the suggested route and the route taken byChristine and Dan on the detour.

Network system 130 aggregates 250 realized routes received from clientdevices 110 that were provided with suggested routes. Aggregatingrealized routes allows network system 130 to determine a likelihood thatspecific realized routes reflect a dissimilarity between the map and thecurrent road network. For example, the construction project in the areais new and Christine and Dan are one of the first transport servicesthat are forced to take a detour. Network system 130 continues toreceive 230, and aggregate 250, realized routes from client devices 110that take a detour when provided a suggested route that includes theconstruction project.

Network system 130 determines 260 a dissimilarity using aggregatedrealized routes received from a plurality of client devices 110 providedthe suggested route. Determining dissimilarities is described in moredetail below. As an example, client devices 110 provided the suggestedroute continue to take detours due to the construction project. As such,many realized routes received from client devices 110 indicate a routedifference between the realized route and the predicted route for thesuggested route. Due to the increased number of route differences causedby the construction project, network system 130 determines 260 thatthere is a dissimilarity between the map used to generate the suggestedroute and the current road network at the location of the constructionproject.

Network system 130 can determine an alternate route for a suggestedroute based on the dissimilarity and received realized routes asdescribed in regards to FIG. 6 . For example, using Christine and Dan’srealized routes, network system 130 can determine an alternate routethat avoids the construction project.

Network system 130 can also determine alternative guidance instructionsfor a suggested route based on the dissimilarity and received realizedroutes as described in regard to FIG. 7 . Consider, for example, thatChristine and Dan miss a turn when traversing a suggested route due to amisleading guidance instruction rather than a construction project. Inthis case, Christine and Dan’s realized routes include a routedifference due to the misleading guidance instruction. In this case,network system 130 may determine that route differences stem from themisleading guidance instructions and can generate modified guidanceinstructions that reduce the number of service providers missing theturn.

Network system 130 can generate a data structure describing a detecteddissimilarity. The data structure may describe a variety of informationabout a detected dissimilarity. The information may include, forexample, a likelihood, a location, a type, a confidence, or any otherinformation describing a determined dissimilarity. In some examples,network system 130 can use aggregated data to determine any number ofdissimilarities for any number of road segments in the road network. Inthis case, network system 130 can output a data structure describing allof the determined dissimilarities in the road network.

Service Coordination

Returning to FIG. 1 , the illustrated environment 100 includes clientdevices 110 and a network system 130 that enable service coordinationvia a network 120, according to one example embodiment. Within theenvironment 100, network system 130 determines dissimilarities usingpredicted routes and realized routes as client devices coordinateservice. FIG. 3 gives a more detailed view of modules within networksystem 130 and FIG. 4 gives a more detailed view of modules within aclient device 110), according to some example embodiments. Someembodiments of client devices 110 and network system 130 have differentmodules than those described herein. Similarly, functionality of theclient devices 110 and network system 130 can be distributed among themodules in a different manner than is described herein.

Network system 130 enables service coordination in the environment 100between client devices 110. Users of network system 130 may includeproviders that provide a service to other users. In an example use case,a provider operates a transportation vehicle to transport a user from afirst location (e.g., pickup location) to a second location (e.g., adestination location). In some cases, the transportation vehicle may beautonomous. Other types of service include, for example, delivery ofgoods (e.g., mail, packages, or consumable items) or services. Althoughthe embodiments herein for facilitating service coordination aredescribed with respect to a transport service, the embodiments hereincan be applied to any type of service that requires a servicecoordination via a network system 130 (e.g., food delivery, shopping,etc.).

As an example, network system 130 enables coordinating transport servicebetween users of client devices 110 within the environment 100. In thecontext of the description, an operator of a transportation vehicle(i.e., the service provider) provides the service of transporting aperson (i.e., the service requestor) to a destination requested by theperson. In one embodiment, transportation vehicle includes all vehiclessuch as cars and motorcycles, as well as public transportation vehiclessuch as trains, light rail, buses, etc. In some cases, thetransportation vehicle can be outfitted with a sensor suite that allowsthe user of client device 110 to obtain information about the roadnetwork, the environment 100, and the operational state of thetransportation vehicle as the transportation vehicle traverses the roadnetwork.

Further, a client device 110 can correspond to a mobile computingdevice, such as a smartphone. In some cases, a client device 110corresponds to an onboard computing system of an autonomoustransportation vehicle. Network system 130 can also correspond to a setof servers and can operate with or as part of another system thatimplements network services. Network system 130 and client devices 110comprise a number of “modules,” which refers to hardware componentsand/or computational logic for providing the specified functionality.That is, a module can be implemented in hardware, firmware, and/orsoftware (e.g., a hardware server comprising computational logic), otherembodiments can include additional modules, can distribute functionalitybetween modules, can attribute functionality to more or fewer modules,can be implemented as a standalone program or as part of a network ofprograms, and can be loaded into memory executable by processors.

In one embodiment, the transportation of a person from a pickup locationto a destination location is referred to as a trip or a transportservice. Referring to FIG. 3 , routing module 310 of network system 130calculates a suggested route from the pickup location to the destinationlocation along a set of road segments within the environment 100 basedon a map of the road segments within the environment 100. In some cases,routing module 310 can calculate a route from the current location of aservice provider to a pickup location.

Routing module 310 can also determine a set of guidance instructions fora suggested route. Guidance instructions are any sort of instructionthat may be presented by a client device 110 to a service providerand/or service requestor to facilitate providing transport service alonga suggested route. Guidance instructions may include, audio cues,textual cues, visual icons, etc. In some examples, routing module 310may receive and/or access guidance instructions from a third partyservice. Network system 130 transmits the guidance instructions toservice providers and/or service requestors when coordinating transportservice.

Additionally, routing module 310 can indicate when a client device 110is using provided guidance instructions when providing a transportservice. As such, realized routes received by network system 130 can be“unguided” or “guided” realized routes. Guided realized routes arerealized routes received from a client device 110 using the guidanceinstructions when coordinating transport service. For example, routingmodule 310 provides a service provider with textual guidanceinstructions for display on their client device when providing transportservice. The service provider allows application 410 to display theguidance instructions to facilitate transport service. Accordingly,routing module 310 labels realized routes received from the clientdevice as guided realized routes. Unguided realized routes are realizedroutes received from a client device not using the guidance instructionwhen coordinating transport service. In a first example, a serviceprovider receives audio guidance instructions along with a suggestedroute. Here, the service provider disables audio playback of guidanceinstructions using application 410 of their client device 110 whenproviding transport service. Instead, the service provider relies onpersonal knowledge of the road network to provide transport service froma first location to a second location. In a second example, the serviceprovider operates the application 410 in an overview mode. Overview modedisplays an entire suggested route rather than just a portion of thesuggested route and does not provide guidance instructions to theservice provider. When providing transport service, the service providermay not completely adhere to the presented suggested route. Accordingly,in both examples, routing module 310 labels realized routes receivedfrom the client device as unguided realized routes.

Generally, routing module 310 calculates transaction costs for transportservices and coordinates those transport services. A transaction cost isa monetary payment from a service requestor to a service provider inexchange for the service provider transporting the service requestor toa destination location. Routing module 310 may calculate the transactioncost for a transport service based on the distance traveled along theroute during the trip (and/or based on a duration of the trip, anyapplicable tolls, fees, etc.). Routing module 310 determines thedistance traveled during the transport service using geographicinformation (e.g., telematics data) received from client devices (e.g.,such as after completion of the transport service). In some examples,other modules within the environment 100 can determine the distancetraveled during the transport service. Further, information from othermodules may be used in determining transaction costs for a transportservice.

Network system 130 receives (e.g., periodically) geographic informationfrom a client device 110 included in a transportation vehicle as thetransportation vehicle moves its position. The geographic informationincludes geographic points that describe a trip of the transportationvehicle. In one embodiment, the geographic information is globalpositioning system (GPS) information. Throughout the description,geographic information is referred to as telematics data, but thedescription herein can be applied to any other type of geographiccoordinate system.

Network system 130 also includes a network datastore 330. The networkdatastore is a storage device (e.g., a hard-drive, etc.) capable ofstoring information to facilitate service coordination. For example, thenetwork datastore 330 may store a map that can be used to determine asuggested route. Network datastore 330 may also store other informationfor determining dissimilarities in a map, such as, for example, historicstate scores, route differences, and trace data, etc. Network datastore330 may also store guidance instructions for use in coordinatingtransport service in the environment 100.

As shown in FIG. 1 , network system 130 is in communication with aservice provider client device 110B and the service requestor clientdevice 110A via a network(s) 120. In one embodiment, the network 120 isthe Internet or any combination of a LAN, a MAN, a WAN, a mobile, wiredor wireless network, a private network, or a virtual private network.While only a single service provider client device 110B and a singleservice requestor client device 110A are shown in FIG. 1 , any number ofclient devices 110 can be in communication with network system 130 viathe network 120. Further, while described as a service requestor clientdevice 110A and a service provider client device 110B, any client device110 within the environment 100 can include functionality for being aservice requestor or a service provider.

Referring to FIG. 4 , a client device 110 is an electronic device (e.g.,a smartphone) that can interact with network system 130 over the network120. As described herein, client devices 110 for service requestors andservice providers are largely similar in that they can include anapplication 410 that facilitates transport service. The application 410includes a telematics module 420 to determine telematics data. Theclient device 110 may also include a device datastore 430 for storingany relevant information that can facilitate transport service in theenvironment 100.

When the client device 110 is a service requestor device 110A, theservice requestor device 110A is used by a person to request a transportservice from a pickup location to a destination location via a clientapplication 410 included in the service requestor device 110A.Application 310 allows the user of the service requestor device 110A tosubmit a transport service request, which network system 130 thenprocesses in order to select an operator of a transportation vehicle asa transport service provider.

According to some examples, the transport service request may include(i) a user identifier (ID), (ii) a pickup location (e.g., a locationidentifier specified by the user), (iii) a destination location, (iv) avehicle type, and/or (v) a requestor location. For example, thetelematics module 420 uses sensors (e.g., a GPS receiver) included inthe service requestor device 110A to determine the requestor location ofthe service requestor device 110A at various instances in time. Therequestor location of the service requestor device 110A may berepresented by a location identifier such as latitude and longitudecoordinates (“location coordinate”).

The current position of the service requestor device 110A is alsoassociated with a time stamp indicating the time and the date in whichthe telematics module 420 measured the current position of the servicerequestor device 110A. In some examples, the pickup location and/or thecurrent location of the service requestor device 110A may be manuallyinputted into the service requestor device 110A by the user of thedevice, such as by selecting a location on a map or in the form of anaddress including at least a street number and street name.

The coordination service, which is implemented by network system 130and/or other servers or systems, can receive the transport servicerequest over the network 120 and can select a service provider for therequestor. In one example, the coordination service can (i) identify apool of service providers that are available to provide the requestedservice and satisfy one or more conditions (e.g., have the specifiedvehicle type, have sensor system, have a preferred route to the pickuplocation, and/or are within a predetermined distance or estimated traveltime away from the pickup location), (ii) select a service provider fromthe pool of service providers, and (iii) transmit an assignment requestto a service provider client device 110B. The invitation can include thepickup location so that the selected service provider can navigate tothe pickup location for initiating the transport service for therequestor. If the selected service provider accepts the assignmentrequest by providing input on the service provider client device 110B,application 410 transmits an assignment acceptance to network system130. Accepting the assignment can associate the service provider and theservice requestor for transport service and network system 130 cannotify the service requestor device 110A accordingly.

When the client device 110 is a service provider client device 110B, itis, generally, an electronic device (e.g., a smartphone) operated by aservice provider and located within the transportation vehicle used tocomplete transport service. The service provider client device 110Bincludes a client application 410. The client application 410 displays,on the service provider client device 110B, information about a tripthat the service provider has agreed to provide, such as the pickuplocation, and/or navigation and/or mapping information instructing theservice provider to travel to the pickup location. As referred toherein, the pickup location may be the location of the service requestordevice 110A or a location specified by the user of the service requestordevice 110A. The destination location may be the location the serviceprovider ends transport service, or a location specified by the user ofthe service requestor device 110A. The client application 410 may alsodisplay, on the service provider client device 110B, the destination forthe assigned trip if provided by the user of the service requestorclient device 110A.

Client devices 110 include a telematics module 420. Telematics module420 uses one or more sensors of client device 110 to identify telematicsdata from the service provider client device 110B and the servicerequestor device 110A. When client application 410 is a transportservice coordination application, telematics module 420 can identify GPSdata from the client device 110 as the service provider navigates atransportation vehicle along one or more road segments and nodes tocomplete a trip. The GPS data of the transportation vehicle (or clientdevice 110) represents the transportation vehicle’s position atdifferent instances in time during a trip. For example, at time T₁,client device 110 can be at a particular GPS location, identified by alocation identifier (e.g., latitude and longitude coordinates) and atime stamp indicative of the time and date when client device 110measured its current position. If the transportation vehicle is moving,at time T₂ client device 110 can be at a different GPS location. In thismanner, client device 110 periodically measures the current position ofthe transportation vehicle (e.g., every three seconds, every fourseconds, etc.) and periodically provides GPS data that is representativeof the position of the transportation vehicle over time to networksystem 130. Alternatively, client device 110 may provide GPS datawhenever new or updated measurements of the current position of thetransportation vehicle are taken or are available.

Determining Route Differences Using Predicted Routes and Realized Routes

Referring to FIG. 3 , dissimilarity module 320 of network system 130 candetermine route differences using predicted routes and realized routes.To illustrate determining a route difference using dissimilarity module320, consider the description of FIG. 5 .

FIG. 5 illustrates a map of a road network including road segments,nodes, realized routes, and predicted routes that can be used todetermine a route difference, according to one example embodiment. Asillustrated, map 500 includes a road network. The road network includesa number of road segments 510 connected by a number of nodes 512. Theroad segments 510 are illustrated as grey bars and the nodes 512 areillustrated as filled circles. While not illustrated, the map 500 alsoincludes connection information for the road segments 510 and nodes 512in the map.

Map 500 can be used to determine a suggested route when network system130 coordinates a transport service. For example, a service requestortransmits a service request for transport service from a first location520 to a second location 522. For convenience, the first location 520and second location 522 are illustrated on coordinates outside the roadnetwork, however, the locations may have coordinates collocated to roadsegments 510 and/or nodes 512.

Routing module 310 determines a suggested route from the first location520 to the second location 522. In this example, the suggested routeincludes suggested road segments 530A, 530B, 530C, 530D, and 530E. Here,the black arrows within the road segments represent the predicted routesfor service providers providing transport service from the firstlocation 520 to the second location 522 using the suggested route.

Network system 130 transmits the suggested route including the suggestedroad segments 530 to client devices 110 of service providers providingtransport service from the first location 520 to the second location522. Network system 130 receives realized routes from client devices 110as service providers provide transport to service requestors from thefirst location 520 to the second location 522.

In this case, assume that map 500 includes incorrect connectioninformation. Specifically, the road network does not allow a left turnat node 512A. As such, service providers cannot traverse suggestedsegment 530C by turning left from suggested segment 530B at node 512A.Instead, service providers continue transport service by traversingactual segments 540A and 540B in lieu of suggested segments 530C and530D, respectively. In other words, the dashed lines within the roadsegments represent realized routes received from service providersproviding transport service from the first location to the secondlocation. Notably, realized routes coincide with predicted routes onsuggested segments 530A, 530B, and 530E.

Dissimilarity module 320 determines there is a route difference for thesuggested route because there are differences between predicted routesand realized routes. Network system 130 can quantify the routedifferences to determine a dissimilarity between map 500 and environment100.

Determining Dissimilarities Between a Map and a Current Road Network

Consistent route differences may be indicative of a dissimilaritybetween a map and the current road network in the environment. Referringto FIG. 3 , dissimilarity module 320 can quantify a route difference todetermine a dissimilarity. Dissimilarity module 320 can quantify a routedifference for every road segment in a realized route to determine adissimilarity, and can quantify a route difference in a variety ofmanners. Generally, dissimilarity module 320 quantifies a routedissimilarity as how often each road segment in a predicted route isexpected occur in a realized route for a suggested route. Forconvenience this quantification may be referred to herein as a statescore. Other quantifications are also possible.

In one example dissimilarity module 320 calculates a state score using atwo-segment approach. In the two-segment approach, dissimilarity module320 defines trace data as a set of paths. Referring to FIG. 5 , a pathis a pair of road segments, (e.g., 530B and 530C) including an incomingroad segment (e.g., 530B) to a node (e.g., node 512A) and an outgoingroad segment (e.g., 530C) from the node (e.g., node 512A). Notably,dissimilarity module 320 defines path for both realized routes andpredicted routes.

Dissimilarity module 320 calculates an expectation ratio R for each pathin a suggested route of a transport service. In one example,dissimilarity module 320 calculates the expectation ratio R as:

$\begin{matrix}{R = \frac{N_{R}}{N_{P}}} & \text{­­­(1)}\end{matrix}$

where N_(R) is the number of times a path occurs in realized routes andNp is the number of times the path occurs in the predicted routes.

Dissimilarity module 320 calculates a bound B for the expectation ratioR. In one example, the dissimilarity module 320 calculates the bound Bas:

$\begin{matrix}{B = \left\{ {R + t_{\beta,n - 1}\frac{s}{\sqrt{N_{P}}}} \right)\mspace{6mu} if\mspace{6mu} R > 0\mspace{6mu}\mspace{6mu} 1 - \frac{1}{\alpha^{N_{P} + 1}}if\mspace{6mu} R = 0} & \text{­­­(2)}\end{matrix}$

where s is the variance of the expectation ratio R, t is the t-score, αis 0.5, and β is 0.75. α and β can be other values.

Dissimilarity module 320 calculates a general state score S_(G) usingthe bound B. In one example, dissimilarity module 320 calculates generalstate score S_(G) as:

$\begin{matrix}{S_{G} = \frac{1}{B}} & \text{­­­(3)}\end{matrix}$

Given this formulation, a high general state score S_(G) indicates thata path is likely to include a dissimilarity. That is, the road networkin the environment 100 is dissimilar from the map 500 when the generalstate score S_(G) is high.

In another example, dissimilarity module 320 calculates a state scoreusing a three-segment approach. The three-segment approach is similar tothe two-segment approach but includes two nodes rather than a singlenode. In the three-segment approach, the outgoing segment of the firstnode is the incoming segment of the second node. In other words, a pathincludes a road segment incoming to a first node, a second road segmentoutgoing from the first node and incoming to a second node, and a thirdroad segment outgoing from a second node.

In both the two-segment and the three-segment approach, dissimilaritymodule 320 calculates an under-constrained expectation ratio Ru and anover-constrained expectation ratio Ro for each path in a suggested routefor a transport service. In one example, dissimilarity module 320calculates the over-constrained expectation ratio Ro as:

$\begin{matrix}{R_{O} = \frac{N_{P}}{N_{R}}} & \text{­­­(4)}\end{matrix}$

and the under-constrained expectation ratio Ru as:

$\begin{matrix}{R_{U} = \frac{N_{R}}{N_{P}}} & \text{­­­(5)}\end{matrix}$

Dissimilarity module 320 calculates an under-constrained bound Bu and anover-constrained bound Bo for the under-constrained expectation ratio Ruand the over-constrained expectation ratio Ro. In one example,dissimilarity module 320 the under-constrained bound Bu and anover-constrained bound Bo similarly to Equation 2.

Dissimilarity module 320 calculates an under-constrained state score Suand an over-constrained state score So for the under-constrained boundBu and under-constrained bound, respectively. In one example, networksystem calculates the under-constrained state score S_(U) and theover-constrained state score So similarly to Equation 3.

A high under-constrained state score S_(U) can be indicative of severaltypes of dissimilarities, such as, for example, paths including extraroad segments, paths including road segments wrongly considered asallowed in one way, and paths including road segments missing a turnrestriction. A high over-constrained state score So can be indicative ofseveral types of dissimilarities, such as, for example, paths includingroad segments with extra turn restrictions.

For convenience, general state S_(G), under-constrained state-score Su,and over-constrained state score So, and may be referred to in aggregateas state score S.

Dissimilarity module 320 may associate a particular state score with astate of a road segment (or path) in a road network. A state of a roadsegment represents the traversability of a road segment or road segmentsincluded in a path. For example, a state score S above a threshold(e.g., 5000) may indicate that a road segment is “closed” while a statescore S below a threshold (e.g., 100) may indicate that a road segmentis “open.” As another example, a state score S above a first thresholdand below a second threshold may indicate that a road segment is“intermittently closed.” As described above particular state scores mayalso indicate various types of dissimilarities.

A state score S for a road segment or path may be stored in networkdatastore 330. Accordingly, dissimilarity module 320 can determinedifferences in a state score S over time. To do so network system mayaccess a previously determined state score(s) (“historic state score”)for a road segment and compare the historic state score to a currentlydetermined state score. In other words, historic state scores are arepresentation of how often a particular road segment is expected tooccur in a realized route for a suggested route. Differences between ahistoric state score and a current state score may indicate a change ina state of a road segment or road segments in a path. For example, aparticular road segment has a high historic state score, and, suddenly,the current state score is low. Network system can use a changing statescore as indicator for a variety of possibilities, such as, for example,(i) the road segment state changed from “closed” to “open” orvice-versa, (ii) a guidance instruction associated with a road segmenthas improved, and (iii) a dissimilarity in a map has been remedied.Other examples are possible.

Dissimilarity module 320 may take various actions to improve accuracy indetermining dissimilarities and route differences. For example,dissimilarity module 320 can filter state scores in a variety of mannersto improve accuracy in determining dissimilarities. As some examples,network system can (i) disregard state scores for U-turns included inrealized routes, (ii) disregard state scores for looping paths (e.g.,traffic circles) included in realized routes, and (iii) for highover-constrained state scores, use route adherence metrics.

Dissimilarity module 320 can filter realized routes in a variety ofmanners to improve accuracy of state scores. As some examples,dissimilarity module 320 can (i) remove realized routes that representallowable paths (i.e., removing false positives), (ii) remove realizedroutes with low horizontal accuracy, (iii) remove realized routesrepresenting path s with below a threshold number of similar paths(i.e., statistically significant sampling). Other example filters arepossible.

Dissimilarity module 320 can filter realized routes in the three-segmentapproach in using several specific approaches to improve the accuracy ofstate scores. For example, dissimilarity module 320 can filter pathshaving road segments that are in intersection of the road network. Morespecifically, dissimilarity module 320 can (i) use paths whose first andsecond road segments are road segments not included in intersections,(ii) remove paths having above a threshold (e.g., 3) number of roadsegments in an intersection, (iii) remove paths having more than athreshold number (e.g., 2) of consecutive in-intersection road segments.Other methods of removing intersections from paths are also possible.

Determining a Misleading Guidance Instruction

In some situations, dissimilarity module 320 can determine a misleadingguidance instruction when a state score indicates a dissimilarity. Inthis case, the state score may indicate cases in which a dissimilaritybetween a map and a road network does not exist, but a significantnumber of route differences do exist. The route differences may becaused by guidance instructions that are misleading. A misleadingguidance instruction can cause a service provider to traverse anincorrect route rather than the suggested route. For example, amisleading guidance instruction may cause service providers to miss aparticular turn in a suggested route.

Dissimilarity module 320 determines a misleading score to quantifyguidance instructions that may be misleading. The misleading scorerepresents a likelihood that a particular guidance instruction causes aservice provider to traverse an incorrect route rather than a suggestedroute. To calculate a misleading score, dissimilarity module 320 maydefine paths for predicted and realized routes using the two-segment orthree segment approach described herein.

Dissimilarity module 320 calculates a path adherence ratio T for eachpath in a suggested route of a transport service. In one example,network system 130 calculates the path adherence T as:

$\begin{matrix}{T = \frac{N_{R}}{N_{P}}} & \text{­­­(6)}\end{matrix}$

where N_(R) is the number of times a path occurs in realized routes andN_(P) is the number of times the path in the predicted routes.

Additionally, when calculating the path adherence ratio T, dissimilaritymodule 320 may filter realized routes based on whether or not therealized route reflects are a reroute of a suggested route. In anexample, the dissimilarity module 320 filters all road segments and/orturns in a suggested route after the first turn of a reroute. Toillustrate, a suggested route includes six road segments. After thethird road segment, a service provider traversing the suggested routestakes a turn not included in the suggested route. Thus, thedissimilarity module filters the fourth, fifth and sixth road segmentsfrom the suggested route when calculating the turn adherence ratio.

Dissimilarity module 320 calculates a guided path adherence ratio T_(g)and an unguided path adherence ratio T_(u). A guided path adherenceratio T_(g) is the path adherence ratio T in which guidance instructionsare presented to a service provider when providing transport serviceusing the suggested route (i.e., using guided realized routes). Anunguided path adherence ratio T_(u) is the path adherence ratio T inwhich guidance instructions are not presented to a service provider whenproviding transport service using the suggested route (i.e., usingunguided realized routes). In one example, the dissimilarity module 320calculates the guided path adherence ratio T_(g) as:

$\begin{matrix}{T_{g} = \frac{N_{R,g}}{N_{P,g}}} & \text{­­­(7)}\end{matrix}$

and the unguided path adherence ratio T_(u) as:

$\begin{matrix}{T_{u} = \frac{N_{R,u}}{N_{P,u}}} & \text{­­­(8)}\end{matrix}$

where the subscript u indicates unguided and the subscript g representsguided. Thus, N_(R),_(g) indicate guided realized routes and N_(R),_(u)represent unguided realized routes.

Dissimilarity module calculates a guided bound B_(g) and an unguidedbound Bu for the guided path adherence ratio T_(g) and the unguided pathat adherence ratio T_(u), respectively. In one example, dissimilaritymodule 320 calculates the guided bound as B_(g):

$\begin{matrix}{B_{g} = \left\{ {T_{g} + t_{\beta,N_{P,g + 1}} \ast \frac{s_{g}}{\sqrt{N_{P,g}}}} \right)\mspace{6mu} if\mspace{6mu} T_{g} > 0\mspace{6mu}\mspace{6mu} 1 - \frac{1}{\alpha^{N_{P,g} + 1}}if\mspace{6mu} T_{g} = 0} & \text{­­­(9)}\end{matrix}$

and calculates the unguided bound Bu as:

$\begin{matrix}{B_{u} = max\left( {0,T_{U} = t_{\beta,N_{P,u - 1}} \ast \frac{s_{u}}{\sqrt{N_{P,u}}}} \right)} & \text{­­­(10)}\end{matrix}$

where s is the variance of the expectation ratio T, t is the t score, αis 0.1, and β is 0.05. α and β can be other values.

Dissimilarity module 320 calculates a misleading score Musing theunguided bound and the guided bound. In one example dissimilarity module320 calculates the misleading score M as:

$\begin{matrix}{M = \frac{B_{g}}{B_{u}}} & \text{­­­(11)}\end{matrix}$

Given this formulation, a low misleading score M indicates that aguidance instruction is likely to misleading and cause a serviceprovider to traverse an incorrect turn. Dissimilarity module 320 candetermine a guidance instruction is misleading if the misleading score Mis above a threshold (e.g., 2000).

Dissimilarity module 320 may take various actions to improve accuracy indetermining misleading guidance instructions. For example, dissimilaritymodule 320 can filter misleading guidance instructions in a variety ofmanners to improve accuracy in determining misleading guidanceinstructions. As some examples, dissimilarity module 320 can (i) onlycalculate a misleading score M for a turn with N_(P),_(u) greater than athreshold number, (ii) only calculate a misleading score M for turnswith N_(P),_(g) greater than a threshold number, and (iii) onlycalculate a mislead score M for turn adherence ratios less than athreshold.

Dissimilarity module 320 may cluster misleading guidance instructions.To do so, dissimilarity module 320 sorts the misleading guidanceinstructions by the type of turn they represent. For example,dissimilarity module 320 may sort guidance instructions for no-turnmaneuvers, U-turns, left turns, etc. Dissimilarity module 320 may thengroup the misleading guidance instructions by the type of guidanceinstruction. For example, dissimilarity module 320 may group themisleading guidance instructions based on maneuver type, deviation ofmaneuver type, etc. Dissimilarity module 320 can determine an impactfactor for misleading guidance instruction based on the number ofmisleading guidance instructions in each cluster. For example, theguidance instruction “turn left” may be less misleading than “veerleft.”

Generating an Alternate Route

FIG. 6 is a flow diagram illustrating a process 600 for determining analternate route for a transport service based on route differences,according to one example embodiment. In this particular example, theroute difference is caused by a change in a road state. In variousembodiments, method 600 may be a particular implementation of method 200of FIG. 2 .

To begin, network system 130 coordinates 610 transport service between aset of service requestors and a set of service providers in anenvironment 100 including a road network. In particular, the set ofservice requestors request transport service from a first location to asecond location. Network system 130 receives the requests and determines(e.g., step 210 of method 200) a suggested route from the first locationto the second location using a map of the road network. The suggestedroute includes a set of road segments that allows service providersoperating a transportation vehicle to provide transport to the servicerequestors from the first location to the second location. Networksystem 130 generates (e.g., step 220 of method 200) a predicted routeincluding the set of road segments from the first location to the secondlocation included in the suggested routes. Network system 130 assignsthe transport services to service providers and transmits the suggestedroute to the service providers. The service providers travel to thefirst location using transportation vehicles. The service requestorsenter the transportation vehicles of the service providers and theservice providers begin providing transport service. The serviceprovider providers transport service by attempting to traverse the roadsegments included in the suggested route. The service providers completetransport service by arriving at the second location.

As the service providers and service requestors travel from the firstlocation to the second location, their client devices 110 transmitrealized routes to network system 130 that reflect road segmentstravelled during the transport service. Network system 130 receives 620the realized routes from the service providers and service requestors.

In this example, a construction project is occurring on a particularroad segment of the suggested route. The construction project has closedthe particular road segment to allow workers to traverse the roadsegment safely (i.e., the road segment has a “closed” state). As such,the service providers are unable to provide transport service using theparticular road segment and, instead, service providers traverse anincorrect route rather than the suggested route to the second location.The incorrect route includes road segments that are not included in thesuggested route, and, therefore, some of the realized routes received bynetwork system 130 include road segments that are not included in thesuggested route. In other words, there is a route difference between therealized routes received from the client devices 110 and the predictedroutes for the suggested route.

Network system 130 determines road segments included in the routedifference between the realized routes and the predicted routes. Thatis, network system 130 determines which road segments are included in apredicted route for a suggested route and not included in the realizedroute for the suggested route (“disparate road segment”). Here, thedisparate road segment is the particular road segment that includes theconstruction project. As described above, network system may define roadsegments in realized routes and predicted routes as paths to determinedisparate road segments.

Network system 130 determines 630 a current state score for thedisparate road segment. The current state score is a quantification ofhow often the disparate road segment occurs in the realized routesreceived from service providers provided the suggested route for thetransport service. To illustrate, network system 130 provides 1500service providers the suggested route such that they may providetransport service from the first location to the second location.Because of the construction project on the particular road segment, 1483of the service providers use an incorrect route while the remaining 5service providers use the suggested route. Accordingly, none of therealized routes received from service providers taking an incorrectroute include the disparate segment while all of the realized routesreceived from service providers taking the suggested route include thedisparate road segment. Thus, in this example, the current state scorefor the disparate road segment is 0.01. The current state score may bedetermined in other manners as described above.

Network system 130 determines 640 (or accesses) a historic state scorefor the disparate road segment. The historic state score is arepresentation of how often the disparate road segment has previouslyoccurred, or is expected to occur, in realized routes of serviceproviders provided the suggested route. For example, before theconstruction project began, nearly every service provider provided thesuggested route provides transport service along the suggested route. Inother words, before the construction began there were very few routedifferences for the suggested route. As a quantitative example, in thepast month 14,356 service providers provide transport service from thefirst location to the second location along the suggested route, while251 service providers take the incorrect route. The realized routesreceived from service providers taking incorrect routes do not includethe disparate road segment while the realized routes received fromservice providers taking the suggested route include the disparate roadsegment. Accordingly, the baseline state score is 0.98 for the disparateroad segment. The baseline state score may be determined in othermanners as described herein.

Network system 130 determines 650 a current state of the disparate roadsegment based on the difference between the historic state score and thecurrent state score. In this example, the disparate road segment has abaseline state score of 0.98 and a current state score of 0.01. Networksystem 130 determines that the current state is “closed” because thedifference between the state scores is above a first threshold (e.g.,0.90) indicating that the state has changed from “open” to “closed.”Network system 130 may store the current state (and current state score)as the historic state and state score if the current state is dissimilarfrom the historic state. Determining the current state score can use theexample approaches as described herein.

Network system 130 determines an alternate route based on thedifferences between the current state and the historic state of thedisparate road segment. In this example, network system 130 determines660 an alternate route because the historic state “open” is differentfrom the current state “closed” for the disparate road segment. As such,network system 130 determines an alternate route from the first locationto the second location that does not include the disparate road segment.

Network system 130 determines 660 the alternate route, rather than thesuggested route, for any subsequent transport service requested from thefirst location to the second location. For example, network systemreceives a transport service request from the first location to thesecond location from a service provider. Network system 130 determines660 the alternate route because the construction project has closed roadsegments along the suggested route.

Network system 130 transmits 670 the alternate route to the serviceproviders such that the service providers can provide transport servicefrom the first location to the second location without having toencounter the construction project.

For ease of description, the above example referenced a single suggestedroute from a first location to a second location. However, similarmethodologies for determining a suggested route apply when usingaggregated realized routes. For example, suggested routes from a varietyof first locations to a variety of second locations may include aparticular road segment. Accordingly, a construction project includingthe particular road segment, for example, can cause route differencesacross various suggested routes. Thus, network system 130 can accessaggregated realized routes and aggregated predicted routes for aparticular road segment when determining a current state score, ahistoric state score, and/or a current state of a road segment. Toillustrate, for example, the construction project may cause severalthousand transport service providers to take incorrect routes forseveral hundred suggested routes. In this case, the disparate roadsegment occurs in realized routes across the several hundred suggestedroutes while the construction project is occurring.

Additionally, for ease of description, the above example referenced aroute difference including a single disparate road segment. However, invarious examples, route differences can include several disparate roadsegments. Network system 130 may determine current state scores,historic state scores, and current states of any number of the disparateroad segments. For example, a construction project includes three roadsegments and a suggested route includes two of the three road segments.In this case, network system 130 can determine that the disparate roadsegments are the two road segments on which the construction projectoccurs that are also included in the suggested route. Network system 130may generate an alternate route that does not includes the two disparateroad segments.

Modifying Guidance Instructions

FIG. 7 is a flow diagram illustrating a process 700 for modifyingguidance instructions for a transport service based on routedifferences, according to one example embodiment. In this particularexample, the route differences are caused by an inaccurate guidanceinstruction. In various embodiments, method 700 may be a particularimplementation of method 200 of FIG. 2 .

To begin, network system 130 coordinates 710 transport service similarlyto step 610 of method 600. That is, a set of service requestors requesttransport service from a first location to a second location, networksystem determines a suggested route for the transport service, networksystem generates a predicted route for the suggested route, and serviceproviders attempt to provide transport service using the suggestedroute. The service providers and service requestors transmit realizedroutes to network system 130 as they coordinate transport service andnetwork system 130 receives 720 the realized routes. The serviceproviders complete transport service by arriving at the second location.

In this example, however, the network system 130 determines (oraccesses) a number of guidance instructions to assist service providersin providing transport service. More particularly, the guidanceinstructions are a set of audio and/or visual cues (e.g., “Turn right in1000 feet.”) to be played back by a client device 110 at certaincoordinates along the suggested route. In some cases, the guidanceinstructions are generated and provided to a client device 110 by athird party system.

Here, a guidance instruction of the set of guidance instructionsprovided to the service providers for the suggested route is misleading.The misleading guidance instruction may cause service providers totraverse an incorrect route rather than the suggested route. Themisleading guidance instruction may say, for example, “Veer right in1000 feet” when traversing the suggested route. However, rather thanveering right to traverse the suggested route, a service provider mustturn right onto a new road segment to traverse the suggested route.Thus, some service providers using the misleading guidance instructionsmay veer right and miss the turn, while other service providers notusing the misleading guidance instructions may correctly turn right ontothe new road segment. As such, some of the service providers traverse anincorrect route that includes a particular road segment not included inthe suggested route. Thus, some of the realized routes received bynetwork system 130 include road segments that are not included in thesuggested route. In other words, there is a route difference between therealized routes and the predicted routes for some of the serviceproviders.

Network system 130 determines disparate road segments for the suggestedroute. As described above, network system may define paths usingrealized routes and predicted routes when determining disparate roadsegments. Here, the disparate road segment is the particular roadsegment traversed because of the misleading guidance instruction. Insome examples, network system 130 only analyzes guided realized routeswhen determining a misleading guidance instruction.

Network system 130 determines 730 a current state score for thedisparate road segment. The current state score is a quantification ofhow often the disparate road segment occurs in the realized routesreceived from service providers provided the suggested route for thetransport service. To illustrate, network system 130 provides 1000service providers the suggested route and corresponding guidanceinstructions for the transport service. Because of the misleadingguidance instruction, 332 of the service providers use an alternateroute while the remaining 728 service providers use the suggested route.Accordingly, none of the realized routes received from service providerstaking the incorrect route include the disparate segment while all ofthe realized routes received from service providers taking the suggestedroute include the disparate road segment. Thus, in this example, thecurrent state score for the disparate road segment is 0.63. The currentstate score may be determined in other manners as described herein.

Because the current state score is below a threshold (e.g., 0.85),network system 130 determines if any of the guidance instructionsprovided to service providers for the suggested route are misleading. Todo so, network system 130 accesses 740 the guidance instructions for thesuggested route that is provided to service providers. In some cases,network system 130 may access the instructions from a third partysystem. Alternatively, if the third-party guidance instructions arereceived by a client device 110 for play-back from a third-party system,application 410 may input the guidance instructions and transmit them tonetwork system 130.

Network system 130 determines 750 the misleading guidance instructioncausing service providers to traverse the disparate road segment. In oneexample, network system 130 determines that the guidance instructionimmediately preceding the disparate road segment is the misleadingguidance instruction. In other examples, network system determines alikelihood (e.g., a misleading score) that a guidance instruction ismisleading. The likelihood may be based on the route differences forservice providers using guidance instructions on a suggested route andservice providers not using guidance instructions on the suggested routeas described herein. Further, the network system 130 may determine alikelihood for any number of guidance instructions preceding thedisparate road segment and select the guidance instruction with thehighest likelihood as the misleading guidance instruction. Toillustrate, using the above example, perhaps the guidance instructionpreceding “Veer right in 1000 feet” is “Follow the left lane ontoHighway 103.” In this situation, a necessary transition from the leftlane to the right lane to achieve the right turn crosses several lanesof traffic. As such, many service providers prompted to “Veer right in1000 feet” miss the turn when trying to cross the lanes of traffic. Inthis case, when accessing aggregate trace data, network system 130determines that the likelihood of the “veer right” guidance instructionbeing misleading is lower than the “follow the left lane” guidanceinstruction. As such, network system 130 determines the “follow the leftlane” guidance instruction is misleading. Many other examples arepossible. Network system 130 may associate an indicator for determinedmisleading guidance instructions in datastore 330. The indicators may beaccessed for by an administrator of network system 130 such that theadministrator may modify the guidance instructions to make them lessmisleading.

Network system 130 modifies 760 the misleading guidance instruction toincrease the number of service providers traversing the suggested routerather than the incorrect route. In other words, network system 130modifies the misleading guidance instruction to decrease the number ofroute differences for a suggested route. Network system 130 can modify760 a misleading guidance instruction in a variety of ways. For example,network system 130 can change the location at which the misleadingguidance instruction is presented to a service provider, modify thelanguage of the misleading guidance instruction, remove the misleadingguidance instruction, replace the misleading guidance instructions, oradd a misleading guidance instruction to the set of guidanceinstructions that lessens the effect of the misleading guidanceinstruction.

Network system 130 stores the modified guidance instruction for futureuse for transport service using the suggested route. For example,network system 130 receives a number of transport service requests fromthe first location to the second location from service providers.Network system 130 determines the suggested route and guidanceinstructions for the transport service. However, in this case, theguidance instructions include the modified guidance instruction(s)rather than the misleading guidance instruction. Network system 130transmits the suggested route and guidance instruction to the serviceproviders such that the service providers can provide transport servicefrom the first location to the second location. Because the guidanceinstructions include the modified guidance instruction, fewer serviceproviders miss the turn and take incorrect routes.

Network system 130 may be configured to continuously improve misleadingguidance instructions. To illustrate, using the previous example, themodified guidance instruction improves the current state score bycausing more people to appropriately take the right turn. However, thecurrent state score remains below the threshold despite a larger numberof service providers appropriately taking the right turn. Accordingly,network system 130 continuously modifies 760 the guidance instructionuntil the current state score is above the threshold.

For convenience, the above example referenced a single suggested routefrom a first location to a second location. However, similarmethodologies for determining a modified guidance instruction apply whenusing aggregated realized routes. For example, suggested routes from avariety of first locations to a variety of second location may include aparticular road segment. For any of the various road segments includedin the suggested routes, guidance instructions may be similar. Forexample, several suggested routes may include a right turn from a firstsegment to a second segment and the guidance instructions for thatparticular right turn are the same between suggested routes. Thus,network system 130 can access aggregated realized routes and aggregatedpredicted routes for a particular road segment when determining acurrent state score, accessing a set of guidance instructions,determining an inaccurate guidance instruction, and/or modifying theinaccurate guidance instruction. To illustrate, for example, aninaccurate guidance instruction may cause several thousand transportservice providers to traverse incorrect routes over several hundredsuggested routes. In this case, the disparate road segment across theseveral hundred suggested routes is the road segment including aninaccurate guidance instruction. Thus, network system 130 can determineand modify the inaccurate guidance instruction for the disparate roadsegment accordingly.

Example Computer System

FIG. 8 is a block diagram illustrating components of an example machinefor reading and executing instructions from a machine-readable medium.Specifically, FIG. 1 shows a diagrammatic representation of networksystem 130 and client device 110 in the example form of a computersystem 800. The computer system 800 can be used to execute instructions824 (e.g., program code or software) for causing the machine to performany one or more of the methodologies (or processes) described herein. Inalternative embodiments, the machine operates as a standalone device ora connected (e.g., networked) device that connects to other machines. Ina networked deployment, the machine may operate in the capacity of aserver machine or a client machine in a server-client system environment800, or as a peer machine in a peer-to-peer (or distributed) systemenvironment 800.

The machine may be a server computer, a client computer, a personalcomputer (PC), a tablet PC, a set-top box (STB), a smartphone, aninternet of things (IoT) appliance, a network router, switch or bridge,or any machine capable of executing instructions 824 (sequential orotherwise) that specify actions to be taken by that machine. Further,while only a single machine is illustrated, the term “machine” shallalso be taken to include any collection of machines that individually orjointly execute instructions 824 to perform any one or more of themethodologies discussed herein.

The example computer system 800 includes one or more processing units(generally processor 802). The processor 802 is, for example, a centralprocessing unit (CPU), a graphics processing unit (GPU), a digitalsignal processor (DSP), a controller, a state machine, one or moreapplication specific integrated circuits (ASICs), one or moreradio-frequency integrated circuits (RFICs), or any combination ofthese. The computer system 800 also includes a main memory 804. Thecomputer system may include a storage unit 816. The processor 802,memory 804, and the storage unit 816 communicate via a bus 808.

In addition, the computer system 800 can include a static memory 806, agraphics display 810 (e.g., to drive a plasma display panel (PDP), aliquid crystal display (LCD), or a projector). The computer system 800may also include alphanumeric input device 812 (e.g., a keyboard), acursor control device 814 (e.g., a mouse, a trackball, a joystick, amotion sensor, or other pointing instrument), a signal generation device818 (e.g., a speaker), and a network interface device 820, which alsoare configured to communicate via the bus 808.

The storage unit 816 includes a machine-readable medium 822 on which isstored instructions 824 (e.g., software) embodying any one or more ofthe methodologies or functions described herein. For example, theinstructions 824 may include the functionalities of modules of thesystem 830 described in FIG. 1 . The instructions 824 may also reside,completely or at least partially, within the main memory 804 or withinthe processor 802 (e.g., within a processor’s cache memory) duringexecution thereof by the computer system 800, the main memory 804 andthe processor 802 also constituting machine-readable media. Theinstructions 824 may be transmitted or received over a network 826(e.g., network 120) via the network interface device 820.

While machine-readable medium 822 is shown in an example embodiment tobe a single medium, the term “machine-readable medium” should be takento include a single medium or multiple media (e.g., a centralized ordistributed database, or associated caches and servers) able to storethe instructions 824. The term “machine-readable medium” shall also betaken to include any medium that is capable of storing instructions 824for execution by the machine and that cause the machine to perform anyone or more of the methodologies disclosed herein. The term“machine-readable medium” includes, but not be limited to, datarepositories in the form of solid-state memories, optical media, andmagnetic media.

Alternative Considerations

The foregoing description of the embodiments has been presented for thepurpose of illustration; it is not intended to be exhaustive or to limitthe patent rights to the precise forms disclosed. Persons skilled in therelevant art can appreciate that many modifications and variations arepossible in light of the above disclosure. For example, while thepresent disclosure discusses predicting provider involvement inpotential safety incidents, the methods and systems herein can be usedmore generally for any purpose where one would want to predictinvolvement in potential incidents using a machine learning model.

Some portions of this description describe the embodiments in terms ofalgorithms and symbolic representations of operations on information.These algorithmic descriptions and representations are commonly used bythose skilled in the data processing arts to convey the substance oftheir work effectively to others skilled in the art. These operations,while described functionally, computationally, or logically, areunderstood to be implemented by computer programs or equivalentelectrical circuits, microcode, or the like. Furthermore, it has alsoproven convenient at times, to refer to these arrangements of operationsas modules, without loss of generality. The described operations andtheir associated modules may be embodied in software, firmware,hardware, or any combinations thereof.

Any of the steps, operations, or processes described herein may beperformed or implemented with one or more hardware or software modules,alone or in combination with other devices. In one embodiment, asoftware module is implemented with a computer program productcomprising a computer-readable medium containing computer program code,which can be executed by a computer processor for performing any or allof the steps, operations, or processes described.

Embodiments may also relate to an apparatus for performing theoperations herein. This apparatus may be specially constructed for therequired purposes, and/or it may comprise a general-purpose computingdevice selectively activated or reconfigured by a computer programstored in the computer. Such a computer program may be stored in anon-transitory, tangible computer readable storage medium, or any typeof media suitable for storing electronic instructions, which may becoupled to a computer system bus. Furthermore, any computing systemsreferred to in the specification may include a single processor or maybe architectures employing multiple processor designs for increasedcomputing capability.

Embodiments may also relate to a product that is produced by a computingprocess described herein. Such a product may comprise informationresulting from a computing process, where the information is stored on anon-transitory, tangible computer readable storage medium and mayinclude any embodiment of a computer program product or other datacombination described herein.

Finally, the language used in the specification has been principallyselected for readability and instructional purposes, and it may not havebeen selected to delineate or circumscribe the inventive subject matter.It is therefore intended that the scope of the patent rights be limitednot by this detailed description, but rather by any claims that issue onan application based hereon. Accordingly, the disclosure of theembodiments is intended to be illustrative, but not limiting, of thescope of the patent rights, which is set forth in the following claims.

What is claimed is:
 1. A method for determining a misleading guidanceinstruction for a suggested route from a first location to a secondlocation in a road network, the method comprising: receiving a pluralityof realized routes, each realized route including a plurality of roadsegments traversed by a client device provided the suggested route fromthe first location to the second location in the road network:determining a state score representing how often a particular roadsegment in the road network occurs in the plurality of realized routes,the particular road segment present in the plurality of road segments ofat least one of the plurality of realized routes; accessing a guidanceinstruction for the suggested route, the guidance instruction comprisinga set of directional instructions provided to the client device fortraversing the suggested route from the first location to the secondlocation; identifying a directional instruction in the set ofdirectional instructions as a misleading directional instruction whenthe state score is below a threshold state score; and modifying theguidance instruction by changing the misleading directional instructionto a new directional instruction predicted to increase the state score.2. The method of claim 1, wherein determining the state score furthercomprises: identifying a first set of realized routes and a second setof realized routes in the plurality of realized routes wherein: thefirst set of realized routes each comprise road segments adhering to theguidance instruction for the suggested route and including theparticular road segment, and the second set of realized routes eachcomprise road segments diverging from the guidance instruction for thesuggested route and not including the particular road segment; andcalculating the state score based on differences between the first setof realized routes and the second set of realized routes.
 3. The methodof claim 1, further comprising: for each realized route, determining anumber of paths from the plurality of realized routes, wherein: eachpath comprises two road segments and one node connecting the two roadsegments from the plurality of realized routes, and determining thestate score uses the number of paths.
 4. The method of claim 1, furthercomprising: for each realized route, determining a number of paths fromthe plurality of realized routes, wherein: each path comprises threeroad segments and two nodes connecting the three road segments, anddetermining the state score uses the number of paths.
 5. The method ofclaim 1, wherein the misleading directional instruction is provided at afirst time when traversing the suggested route and the new directionalinstruction is provided at a second different than the first time whentraversing the suggested route.
 6. The method of claim 1, wherein themisleading directional instruction is provided at a first position whiletraversing the suggested route and the new directional instruction isprovided at a second position while traversing the suggested route, andthe second position is different than the first position.
 7. The methodof claim 1, wherein the misleading directional instruction is thedirectional instruction from the set of directional instructionsprovided before traversing the particular road segment.
 8. The method ofclaim 1, wherein the new directional instruction is an absence of adirectional instruction.
 9. The method of claim 1, wherein identifyingthe misleading directional instruction further comprises: for eachdirectional instruction in the set of directional instructions,calculating a likelihood the directional instruction causes theparticular road segment to be traversed by the client device; andselecting the directional instruction in the set of directionalinstructions with a highest calculated likelihood as the misleadingdirectional instruction.
 10. The method of claim 1, further comprising:receiving the guidance instruction from a third-party system; andtransmitting the modified guidance instruction to the third-partysystem.
 11. A non-transitory computer readable storage medium comprisingcomputer program instructions for determining a misleading guidanceinstruction for a suggested route from a first location to a secondlocation in a road network, the computer program instructions, whenexecuted by one or more processors, causing the one or more processorsto: receive a plurality of realized routes, each realized routeincluding a plurality of road segments traversed by a client deviceprovided the suggested route from the first location to the secondlocation in the road network: determine a state score representing howoften a particular road segment in the road network occurs in theplurality of realized routes, the particular road segment present in theplurality of road segments of at least one of the plurality of realizedroutes; access a guidance instruction for the suggested route, theguidance instruction comprising a set of directional instructionsprovided to the client device for traversing the suggested route fromthe first location to the second location; identify a directionalinstruction in the set of directional instructions as a misleadingdirectional instruction when the state score is below a threshold statescore; and modify the guidance instruction by changing the misleadingdirectional instruction to a new directional instruction predicted toincrease the state score.
 12. The non-transitory computer readablestorage medium of claim 11, wherein the computer program instructionscausing the one or more processors to determine the state score, whenexecuted, further cause the one or more processors to: identify a firstset of realized routes and a second set of realized routes in theplurality of realized routes wherein: the first set of realized routeseach comprise road segments adhering to the guidance instruction for thesuggested route and including the particular road segment, and thesecond set of realized routes each comprise road segments diverging fromthe guidance instruction for the suggested route and not including theparticular road segment; and calculate the state score based ondifferences between the first set of realized routes and the second setof realized routes.
 13. The non-transitory computer readable storagemedium of claim 11, wherein the computer program instructions, whenexecuted by the one or more processors, further cause the one or moreprocessors to: for each realized route, determine a number of paths fromthe plurality of realized routes, wherein: each path comprises two roadsegments and one node connecting the two road segments from theplurality of realized routes, and determining the state score uses thenumber of paths.
 14. The non-transitory computer readable storage mediumof claim 11, wherein the computer program instructions, when executed bythe one or more processors, further cause the one or more processors to:for each realized route, determine a number of paths from the pluralityof realized routes, wherein: each path comprises three road segments andtwo nodes connecting the three road segments, and determining the statescore uses the number of paths.
 15. The non-transitory computer readablestorage medium of claim 11, wherein the misleading directionalinstruction is provided at a first time when traversing the suggestedroute and the new directional instruction is provided at a seconddifferent than the first time when traversing the suggested route. 16.The non-transitory computer readable storage medium of claim 11, whereinthe misleading directional instruction is provided at a first positionwhile traversing the suggested route and the new directional instructionis provided at a second position while traversing the suggested route,and the second position is different than the first position.
 17. Thenon-transitory computer readable storage medium of claim 11, wherein themisleading directional instruction is the directional instruction fromthe set of directional instructions provided before traversing theparticular road segment.
 18. The non-transitory computer readablestorage medium of claim 11, wherein the new directional instruction isan absence of a directional instruction.
 19. The non-transitory computerreadable storage medium of claim 11, wherein the computer programinstructions causing the one or more processors to identify themisleading directional instruction, when executed, further causes theone or more processors to: for each directional instruction in the setof directional instructions, calculate a likelihood the directionalinstruction causes the particular road segment to be traversed by theclient device; and select the directional instruction in the set ofdirectional instructions with a highest calculated likelihood as themisleading directional instruction.
 20. The non-transitory computerreadable storage medium of claim 11, further comprising: receiving theguidance instruction from a third-party system; and transmitting themodified guidance instruction to the third-party system.